Unifying formalism for multivariate information-related measures: Möbius operators on subset lattices

Information-­‐‑ related measures are useful tools for multi-­‐‑ variable data analysis, BLOCKIN BLOCKIN as BLOCKIN BLOCKIN measures BLOCKIN BLOCKIN of BLOCKIN BLOCKIN dependence BLOCKIN BLOCKIN among BLOCKIN BLOCKIN variables, BLOCKIN BLOCKIN and BLOCKIN BLOCKIN as BLOCKIN BLOCKIN descriptions BLOCKIN BLOCKIN of BLOCKIN BLOCKIN order in biological and physical systems. Information-­‐‑ related measures, like marginal entropies, mutual / interaction / multi-­‐‑ information, have been used in a number of fields including descriptions of systems complexity and biological data analysis. The mathematical relationships among these measures are therefore of significant interest. Relations between common information measures include the duality relations based on Möbius inversion on lattices. These are the direct consequence of the symmetries of the lattices of the sets of variables (subsets ordered by inclusion). While the mathematical properties and relationships among these information-­‐‑ related measures are of significant interest, there has been, to our knowledge, no systematic examination of the full range of relationships and no unification of this diverse range of functions into a single formalism as we do here. In this paper we define BLOCKIN BLOCKIN operators BLOCKIN BLOCKIN on BLOCKIN BLOCKIN functions BLOCKIN BLOCKIN on BLOCKIN BLOCKIN these BLOCKIN BLOCKIN lattices BLOCKIN BLOCKIN based BLOCKIN BLOCKIN on BLOCKIN BLOCKIN the BLOCKIN BLOCKIN Möbius BLOCKIN BLOCKIN inversion BLOCKIN BLOCKIN idea that map the functions into one another (Möbius operators.) We show that these operators form a simple group isomorphic to the symmetric group S3. Relations among the set of functions on the lattice are transparently expressed in terms of the operator BLOCKIN BLOCKIN algebra, BLOCKIN BLOCKIN and, BLOCKIN BLOCKIN applied BLOCKIN BLOCKIN to BLOCKIN BLOCKIN the BLOCKIN BLOCKIN information BLOCKIN BLOCKIN measures, BLOCKIN BLOCKIN can BLOCKIN BLOCKIN be BLOCKIN BLOCKIN used BLOCKIN BLOCKIN to BLOCKIN BLOCKIN derive BLOCKIN BLOCKIN a wide range of relationships among measures. We describe a direct relation between sums of conditional log-­‐‑ likelihoods and previously defined dependency measures. The algebra is naturally generalized which yields more extensive relationships. This formalism BLOCKIN BLOCKIN provides BLOCKIN BLOCKIN a BLOCKIN BLOCKIN fundamental BLOCKIN BLOCKIN unification BLOCKIN BLOCKIN of BLOCKIN BLOCKIN information-­‐‑ related BLOCKIN BLOCKIN measures, BLOCKIN BLOCKIN but isomorphism of all distributive lattices with the subset lattice implies broad potential application of these results.